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Abstract. Here we derive a nonsmooth maximum principle for optimal con- 
trol problems with both state and mixed constraints. Crucial to our develop- 
ment is a convexity assumption on the "velocity set" . The approach consists 
of applying known penalization techniques for state constraints together with 
recent results for mixed constrained problems. 

1. Introduction. In this paper we develop a nonsmooth maximum principle for 
optimal control problems with both pure state and mixed state control constraints 
in the presence of a convexity assumption. The problem of interest is 

Minimize l(x(a),x(b)) 
subject to 
±(t) = f(t,x(t),u(t)) &.e.te{a,b] 

h(t,x(t)) < for all t G [a,b] 

(x(t),u(t)) e S(t) a.e. t e [a,b] 

(x(a),x(b)) G E. 



(P) 



The state x and control u are subject to joint, or mixed constraints through the 
condition (x{t),u(t)) G S(t) where t -> S(t) C R" x R k is a multifunction. The 
function / : KxPxt* -> K™ describes the system dynamics and h: [a, b] x R™ — >• R 
is the functional defining the pure state constraint. Furthermore, the closed set 
E C R™ x R™ and (:R"xl"->l specify the endpoint constraints and cost. 

This problem involves measurable control functions u and absolutely continuous 
function x. A pair (x, u) is called an admissible process if it satisfies the constraints 
of the problem with finite cost. 

We say that the process (x,u) is a strong local minimum if, for some e > 0, 
it minimizes the cost over admissible processes (x,u) such that \x(t) — x(t)\ < 
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e for all t G [a, b\. We consider the basic hypotheses on the problem data through- 
out. They are the following: / and L arc C x B n+k , S is C x B, E is closed and I 
is locally Lipschitz. 

Necessary optimality conditions for nonsmooth problems with pure state con- 
straints have been studied systematically for quite some time (sec [22] details and 
references therein). On the other hand, problems with mixed state control con- 
straints, amply studied in a smooth framework (see, for example, [1], [2], [7], [14], 
[15], [16], [17], [19], [20]) have received little attention. Attempts to treat mixed 
constrained problems with nonsmooth data have been in general timid (see, for 
example, [13], [12]) until quite recently when, in [8], necessary conditions in the 
form of a nonsmooth maximum principle were developed. However the literature 
on nonsmooth maximum principle with both mixed and state constraints has been 
surprisingly sparse. 

In this paper wc develop a nonsmmoth maximum principle for problem (P) 
with both pure state and mixed state-control constraints under some convexity 
assumptions. To achieve our purpose we intertwine established approaches used for 
state constraints with up to date developments for problems with mixed constraints. 
Indeed, we follow closely the approach developed in [21] (see also [10] and [12]) where 
necessary conditions for pure state constrained problems are derived. Our proofs 
differ from those in [10] since we deal not only with state constraints but also with 
mixed constraints. So applications of a nonsmooth maximum principle for mixed 
constrained problems, derived in [8] (instead of those in [9]), play a crucial role in 
our analysis. There is however a price to pay; here we assume that the solution of 
(P) is a strong minimum in contrast with [8] where a weaker notion of minimum, 
that of local minimum of radius R is used (in this respect see also [6]). Also we 
need to strengthen the hypotheses in comparison with those in [8]. The convexity 
assumption we impose on this paper may be seen as a major hindrance to some 
applications. Although this assumption can be successfully removed following the 
lines of [11] we opt, for the sake of simplicity, to report that work elsewhere together 
with a discussion of the hypotheses and illustration of applications. 

2. Preliminaries. For g in R m , inequalities like g < are interpreted component- 
wise. Here and throughout, B represents the closed unit ball centered at the origin 
regardless of the dimension of the underlying space and | • | the Euclidean norm or 
the induced matrix norm on M pX9 . The Euclidean distance function with respect 
to a given set A c K m is 

d A : R k -> K, y i-> d A {y) = inf {\y - x\ : x e A} . 

A function h: [a,b] -t W p lies in W 1,1 ([a, if and only if it is absolutely 

continuous; in i 1 ([a, 6]; M p ) iff it is integrablc; and in L°°([a, b];M. p ) iff it is essentially 
bounded. The norm of L°°([a, b];W) is H-]^. 

The space C* ( [a, b] ; E) is the topological dual of the space of continuous functions 
C([a, i>];R). Elements of C*([a, 6]; R) can be identified with finite regular measures 
on the Borel subsets of [a,b]. The set of elements in C*([a, 6]; R) taking nonncgativc 
values on nonnegative- valued functions in C([a,6];R) is denoted by C® ([a, b]; R). 
The norm in C®([a, 6];R), , coincides with the total variation of [i, Jj a ^ fi(ds). 
The support of a measure //, written supp{/i}, is the smallest closed set A c [a, b] 
such that for any relatively open subset B C [a, b]\A we have n(B) = . 
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We make use of standard concepts from nonsmooth analysis. Let A C R fc be a 
closed set with x G A. The proximal normal cone to A at x is denoted by N^(x), 
while N^(x) denotes the limiting normal cone and N^(x) is the Clarke normal 
cone. 

Given a lower semicontinuous function /: M. k — > K U {+00} and a point x G M. k 
where f(x) < +00, d L f(x) denotes the limiting subdifferential of / at x. When the 
function / is Lipschitz continuous near x, the convex hull of the limiting subdiffer- 
ential, co d L f(x), coincides with the ( Clarke) subdifferential d c f(x). For details on 
such nonsmooth analysis concepts, see for example [4, 5, 18, 22]. 

3. Auxiliary Results. In this section we present a simplified version of one of the 
main results in [8] that will be of importance in the forthcoming developments. 
Take a fixed interval [a, b] and a set S of [a, b] x R" x R fc . Define 

S(t) := {(x, u) : (t, x, u) G S} for all t G [a, b}. (1) 

Assume for the time being that E C R" x R" and I : R" x R" -> R. Consider the 
following problem: 

Minimize l(x(a),x(b)) + / L(t,x(f),u(t))dt 

J a 

lr n I subject to 

^ ^ I ±{t) = f(t,x{t),u{tj) &.e.te[a,b] 

(x{t),u(t)) G S(t) a.e. * G [a,b] 
t (x(o),x(6)) G £. 

where X : [a, J] x R" x R l -> R. 
For some £ > 1 define 

Sl(t)={(x,u) G : |a;-2(t)| < e} . 

In generic terms we assume that a function <j)(t, x, u) satisfies [L%] if: 
[LJ] There exist constants k% and fc* such that for almost every t G [a, 6] and every 
(xi,Ui) G <S*(t) (i = 1,2) we have 

\<p(t,xi,ui) - 4>(t 1 x 2l u 2 )\ < k$\xi - x 2 \ + fc*K - u 2 \. 

If this assumption is imposed on /, then the Lipschitz constants are denoted by 
k[ and k[. As for S(t) we consider the following bounded slope condition: 
[BSJ] There exists a constant k$ such that for almost every t G [a, b] the following 
condition holds 

(x,u) G St(t), {a, 13) G N^ t) (x,u) =s- |a| < fc s |/?|. 

The two previous hypotheses are strengthening of the analogous hypotheses in 
[8] . For the sake of uniformity and the analysis in the forthcoming sections we need 
to position an extra hypothesis on the set S*(i). We assume that: 
[CS»] The set S%(i) is closed and there exists an integrable function c such that for 
almost every t G [a, b] the following holds 

S'(t) is closed and (x,u) G Sl(t) =>\( x ,u)\< c{t). 



The £ here can be taken to be equal to the parameter defining the strong local minimum. 
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We observe that although [CSl] is a strong assumption it is nevertheless of 
importance in our future development. Necessary conditions of optimality for (C) 
are given by the following theorem: 

Theorem 3.1. (adaption of Theorem 7.1 in [8]) Let (x,u) be a strong local mini- 
mum for problem (C). Assume that the basic hypotheses, that f and L satisfy [LI] 
and that [BSl\ and [CSl] hold. 

Then there exist an absolutely continuous function p: [a,b] — > K n , and a scalar 
Xq > such that 

(p(t),Ao)^0 VtG[o,6], (2) 
e d^ u [(p(t)J-(t)) - \ L(t) ~ K\p(t)\d s(t) (t)] a.e. (3) 

(x(t), u) G S(t) => (p(t), f(t, x(t),u)) - X L(t, x(t),u) (4) 
< (p(t),f(t,x(t),u(t)) - X L(t,x(t),u(t))) a.e. 

(p(a),-p(b)) G N%(x(a),x(b)) + X dl(x(a),x(b)), (5) 

where f(t) and L{t) represent the function evaluated at (t,x(t),u(t)), ds(t)(t) is 
the distance function to S(t) evaluated at (x(t),u(t)) and K in (3) is a constant 
depending only on k£, k%, k^ and ks- 

4. The Convex Case. We now turn to problem (P). For this problem we derive 
a nonsmooth maximum principle under the following convexity assumption on the 
"velocity set" : 

[C] The velocity set {v € M. n : v = f(t,x,u), u G S(t, x)} is convex for all t G 
[a, b}. 

Furthermore we need to impose two more hypotheses on the data of our problem, 
one related to the state constraint and another to mixed constraints. 

[HI] For all x £ x(t) + eM the function t — s- h(t, x) is continuous and there exists a 
scalar kh > such that the function x ~ ¥ h(t, x) is Lipschitz of rank kh for all 
t £ [a,b\. 

[H2] For almost every t £ [a, b] the following condition holds: for all u £ S(t, x(t)) 
and all sequence x n — > x(t) there exists a sequence u n £ S(t,x n ) such that 
u n — > u. 

In the above the set S(t, x) is defined as 

S(t,x) = {u : (x,u) £ S{t)} 

where S(t) is as in (1). For a discussion on the need to impose continuity of t — > h 
see [10]. Hypothesis [H2] asserts the lower semi- continuity of the multifunction 
x — >• S(t,x) (for definition and properties see [3]). 

Assume the basic assumptions. Also suppose that / satisfies [LI] and that both 
[BS%] and [CSl] hold. Under these assumptions we note for future use that the 
following conditions are satisfied: 

\f(t,x(t),u) - f(t,x(t),u(t))\ < kl\u-u(t)\ for all u £ S(t,x(t)) a.e. t (6) 

for all u £ S(t,x(t)) a.e. t £ [a,b] and there exists an integrable function k such 
that 

\f(t,x(t),u)\ < k(t) for all u £ S(t,x(t)) a.e. t. (7) 
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Before proceeding we need to define the following subdiffcrcntial 

B x h(t,x) := co {lim£j : & € d x h(U,Xi), (U,Xi) -> (8) 
We are now in position to state our main result. 

Theorem 4.1. Lei (S, u) 6e a strong local minimum for problem (P). Assume that 
the basic hypotheses, [C], [HI], [H2], [BSl] and [CSf\ hold and that f satisfies [L%]. 
Then there exist an absolutely continuous junction p: [a,b] — > W 1 , an integrable 
function 7 : [a, b] — > M. n , a measure /1 € C®([a, 6]; M), and a scalar Xq > such that 

(i) /*{[a,6]} + INIoo + A >0 ) 

(ii) (-p(t),0) e d^ u (q(t)J(t,x(t),u(t)))-N^ t) (x(t),u(t)) a.e., 

(iii) (2(f), u) G S(t) => (q(t), f(t, x{t), «)) < (9(t), /(i, i(t), u(i))) a.e., 

(iv) (p(o), -<?(&)) € iV|(«(a), 5(6)) + A 9Z(x(a), 2(6)), 

(v) 7(i) G dh(t,x(t)) pL-a.e., 

(vi) supp{^} C{te [a, b] : h (t, x(t)) = 0} , 

where 

M _ j + /[,,t) 7(»)Md») i e [a, 6) 
91 j ~ \ p(t) + / [o , 6] 7(^(d») * = &■ [ ) 

Remark 1. The proof shows that we prove a sharper form of (ii): 

(-p(t),0) G d^Jq(t),f(t,x{t),u(t))) -fr| 9 (i)|^ u d s(t) (i(t),i2(t)). 

Remark 2. It is also easy to deduce from the proofs that when assumption [H2] is 
not imposed, a "weaker" version of the necessary conditions for (P) (in the vein of 
[9]) can be obtained: all the conclusions but (iii) (the Weierstrass condition) hold. 

We derive Theorem 4.1 in two main stages. In the first stage we establish the 
validity of the theorem to the following problem 



(Q) 



Minimize l(x(b)) 
subject to 

±(t) = 
(x(t),u(t)) G 
h(t,x(t)) < 
(x(a),x(b)) G 



f{t,x(t),u(t)) 

S(t) 



{x a } x E b . 



a.e. t e [a, 6] 
a.e. t £ [a, b] 
for all t G [a, 6] 



Problem (Q) is a special case of (P) in which P = {x a } x P& and l(x a , xj,) = l(xb)- 

Proposition 1. Let (x,u) be a strong local minimum for problem (Q). Assume 
the basic hypotheses, [C], [HI], [BS%\ and [CSl] hold, that Et is closed and that f 
satisfies [L%\. Then there exist an absolutely continuous function p: [a,b] — > K™, an 
integrable function 7 : [a, b] — > M. n , a measure fj. G C®([a, 6]; M), and a scalar Xq > 
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such that 

fx{[a,b]} + \\p\\ oo + X >0, (10) 
(-p(t),0) e dC u (q(t),f(t,x(t),u(t))) - JVg t) (x(t),«(t)) a.e. (11) 
(x(t), u) g S(t) => <«(*), /(*, x(t),u)) < (q(t),f(t, x(t),u(t))) a.e. (12) 
-q(b)eN% b (x(b)) + \ dl(x(b)), (13) 
7(*) G dh(t,x(t)) fi-a.e., (14) 
supp{/4 C{te [a, 6] : ft (t, x(t)) = 0} , (15) 

where q is as in (9). 

5. Proof of Proposition 1. We now proceed proving Proposition 1. Observe that 
u(t) G <S(i, is equivalent to (x(t),u(t)) G 

The local minimality of (x, u) provides some e > 0. By reducing this constant if 
necessary, we can also rely on the hypotheses. The proof breaks into several steps. 
Step 1: Penalize state- constraint violation. 

Define the following problem for each i G N: 

Minimize l(x(b)) + i j h + (t,x(t))dt 

J a 

subject to 

(Qi) \ x(t) = f(t,x(t),u(t)) &.e.t€[a,b] 
(x(t),u(t)) G S(t) a.e. t G [a,b] 
(x(a),x(b)) G {x a } x E b , 

where h + (t,x) := max{0, h(t, x)}. This differs from (Q) by shifting the state con- 
straint into the objective function. 

Following the approach in [21] (see also [10]) let us temporarily assume that 
penalization is effective, i.e., 
[IH] lim infjPj = inf{P}. 

i— y oo 

We will justify this assumption later. 
Step 2: Application of Ekeland's theorem. 

Let W denote the set of measurable functions u : [a, b] — > M. k for which there exists 
an absolutely continuous function x such that x(t) = f(t,x(t),u(t)), (x(t),u(t)) G 
S(t), for almost every t e [a,b], x(t) e x(t) + eM for all t G [a,b], x(a) = x a 
and x(b) £ E b . We provide W with the metric A(u,w) :=|| u — w and define 
Ji : W — > R using the arc x mentioned above: 

Ji(u) :=l(x(b))+i [ h + (t 7 x(t))dt. 



It is a simple matter to check that (W,A) is a complete metric space in which the 
functional J; : W — > R is continuous (see [4]). Moreover, problem (Qj) above is 
closely related to the abstract problem 

Minimize Ji{u) 
subject to u G W. 



(Ri) 



Clearly (u,x(b)) is admissible for (Ri), with Ji(u) = l(x(b)) = inf P since for all 
t G [a, 6], ft. + (i, x(i)) =0 . Let = J{(u) — inf P,. We have £i > and, taking into 
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account [IH], Si — > 0. Ekeland's variational principle (see [22]) applies. It asserts 
the existence of Ui G W such that 

|| Ui-U\\ L1 < y/Ti (16) 

and Ui minimizes over W the perturbed cost functional 

u i-» Ji(u) + yfel || u.i - u \\ Ll . (17) 

Let Xi be the trajectory corresponding to Ui. 

Step 3: Study optimality conditions for the perturbed problem. 

In control-theoretic notation, our work with Ekeland's Theorem shows that the 
process (xi,Ui) solves the following optimal control problem: 

Minimize l(x(b)) +i / h + (t,x(t)) dt + -Jell \u(t) - Ui(t)\ dt 



(A) 



subject to 

x(t) = f(t,x(t),u{t)) a.e. te[a,b] 

(x(t),u(t)) G S(t) a.e. te[a,b] 

x(t) G x(t) + eM for all t G [a, b] 

x(a) = x a 
x(b) G E b . 

Since Ei — > (by [IH]) it follows from (16) that u, — >• u strongly. We can then 
arrange by subsequence extraction, if necessary, that Ui — > u almost everywhere. 
We can further deduce that x% — > x uniformly. By discarding initial terms of the 
sequence we can guarantee that (xj, it,) is a local minimum for a variant of problem 
(Di) obtained by dropping the constraints x(t) G S(t)+eB. We now fix our attention 
in the related subsequence of problems without relabeling. 

Theorem 3.1 applies to (Di). It provides an absolutely continuous function pi 
and a scalar Xi > such that 

(piCtJ.AO^Oforallt, (18) 

(-Pi(t),0) G ^{(piW./Ct.XiW.UiC*))) -tA*h + (* ) au(*)) , in . 

_ (19) 
-7e7A l |u(i) - Ui(t)\ ~ K\pi(t)\d S (t)(3:i(t),Ui(t))} a.e. 

(Pi(t),f(t,%i(t),u)) - VffiAj |u(t) - Ui(t)| < (pi(t)J(t,Xi(t),Ui(t))) a.e. 

-ft(6) G N^( Xi (b)) + Xid L l(xi(b)) (21) 

These conditions have consequences we now seek to express in terms of the orig- 
inal problem (Q). 

Apply Clarke's sum rule [4] to (19) and take into accounts the properties of the 
subdiffcrcntials of the distance function. We deduce that there exist measurable 
functions 7,-, a, <p i an d fi such that for almost every t in [a, b], 

(t l (t),Q(t))£dgj(t,x l (t),u l (t)), (22) 

( 7i (t),0)ed^ u h+(t,Xi(t)), (23) 

e;(i) G R k such that |e. t | < 1, (24) 

(Mt),<Pi(t))£dC u d s{t )(xi(t),Ui(t)), \(<t>i(t),Vi(t))\ < 1 (25) 

such that 



(26) 
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= Pi(t)Ci(t) - ^T t X l e t - K\ Pi (t)\<pi(t). (27) 
To simplify this further, let ho(t,x) = and h\{t,x) = h(t,x) so that 
h + (t, x) = m&x{hj(t,x) : j = 0,1} . 
Then for each fixed t, Clarke's Max Rule [4] says 

d° tU h+(t,Xi(t)) 

C coU} =0 {<?;„!, Al.xMu : h^x^t)) = h+ (t, x t (t))} . 

Clearly d^ u h = {(0,0)}, so a typical element of the right side has the form 
a i (lit 0) , where (74, 0) € d^ u hi(t,Xi(t)) and 014 is chosen from 

E<(i) = {a e[0,l], a = if h^x^t)) < h + {t,Xi(t))} . 
Tracking these dependencies leads to the following expansion of (26) and (27): 
-Pi(t) = Pl (t)t l (t) - iAiOi^W - K\pi{t)\^{t), 

(28) 

= Pi(t)d(t) - ^FiXMt) - K\pi{t)\ifi{t). 
We now introduce the measure /Uj £ C*([a, 6];R): 

dni(t) = / iX l a i (t)dt 
Jb 

for every Borcl set B C [a,b]. Define m £ C*([a,b];R) as dir^t) — Pi(t)dt. Then, 
from (28) we get 



(29) 



and we have 

Pi(t) = k + d-Ki{t) for all t G (a, 6], (30) 

J[a,t) 

for every Borel set B. Here bi = Pi(a). Taking (21) into account we have 

dn(t) € N% b ( Xi (b)) + X l d L l{x l {b)). (31) 

0,6] 

Since a^(t) € Ej(t), we have € C®([a, 6];K) and this measure has support in 
{i£ [a, 6] : h(t,x t (t)) > 0}. 

Since, by (18) bi and Ai are not both zero, we may conclude, after rescaling, that 

\b i \ + \fi i \+X i = l. (32) 

Step 4: Take limits. Our first steps has dealt with fixed i 6 N. We now consider 
the case when i — > 00. Recall that the sequence a;, converges uniformly to x and 
Ui — > ti almost everywhere. 

Under the hypotheses and appealing to Gronwall's inequality we deduce the 
existence of a constant K\ such that \iri\ < K\. It follows from (30) and (32) that 
\pi(t)\ < K\ + 1. We now deduce from the above that 

TTi — > 7r weakly* 

for some measure tt. Consequently 

Itr! Id. 



dTTi(t)= / (^(t)^{t)-K\pi(t)\4>i(t))ltt- / 7i{t)dfli{t) 

J b Jb 

= / (M*)Ci(*)- V^W*)-#biWIPiW)<& 
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Turning again to (32) we may arrange that bi — > 6, A, — s- A, ^ —> fx weakly* for 
some b G W 1 , A > and some measure \x. Also we have — > \/x\ and 

|6| + |A| + H = 1. 

With the above and appealing to Lemma 4.3 in [21] we can now conclude that there 
exists some subsequence such that Pi(t) — > q(t) a.e. where q is now a function of 
bounded variation defined as 



q(t) := b + dn and bi + / dir-i — > b + / dir. 

J[a,t) J[a,t) J[a,t) 

Under the hypotheses we deduce from (22) that |(£i(t), Ci(*))| < max{fc|, k[} a.e. 
Dunford-Pettis Theorem (see for example [22, Theorem 2.51]) asserts existence of 
a subsequence converging weakly in the L 1 topology to some function (£, £) such 
that £, ( E L 1 . Taking into account (24) and (25) we deduce in the same way 
that e,; — > e, (<fii,tpi) — > (<fi,<p) for some e, <f>, (f € L 1 where the convergent 
is understood in the weak L 1 topology. Upper semi-continuity properties of the 
subdiffcrcntials asserts that (22)-(25) hold when we remove the indexes i. 

Observe that d£h(t,x) C B x h(t,x) (see (8) for definition of 3 x h(t,x)) and that 
d x h(t, x) is of closed graph for any i. It follows from [21. Lemma 4.3] that there exists 
a Borcl measurable, /i-intcgrablc function 7 such that ^(t) £ d x h(t,x(t)) p, — a.e. 
This is (14) of the proposition. 

We now turn to (31). The properties of limiting normal cones and limiting 
subdifferential assert that 

dn(t) G N% b (x(b)) + \d L l{x{b)). (33) 

a,b] 

We concentrate on the support of the measure \x. Mimicking the arguments in 
[10] it is a simple matter to see that supp{/x} C {< € [a, b] : h(t 7 x{t)) = 0} . This is 
conclusion (14) of the proposition. 

By [H2] we deduce from (20) that 

(q(t)J(t,x(t),u)) < (g(t), /(*,*(*)»«(*))>■ (34) 
Next we focus on (29). Lemma 4.3 in [21] and our conclusions above assert that 

-q(t) + b = [ (q(s)t(s)-K\q(s)\<t>(sj)ds- [ j(s)dfx(s) 

J\a,t) K 7 J[a,t) 

{q(s)as)-K\q(s)\<p(s))ds 

a,t) K ' 

Define now the function p(t) := q(t)— / r y(s)d/i(s). From the above we now obtain 

J[a,t) 

the conclusions of the proposition. Observe that (10) follows from \b\ + |A| + = 1. 
Step 5: Show that [C] implies [IHJ. We omit the details since the conclusion can 
be obtained adapting the arguments in [10]. 

6. Sketch of the Proof of Theorem 4.1. The proof comprises three stages. We 
omit the details. We first extend Proposition 1 to problems where x(a) £ E a and E a 
is a closed set. This is done following the lines in the end of the proof of Theorem 3.1 
in [21]. Thus we obtain necessary conditions when (x(a),x(b)) € E a x E\,. Next we 
consider the case when the cost is I = l(x(a), x(b)). This is done using the technique 
in Step 2 of section 6 in [12]. And finally, following the approach in section 6 in 
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[12], we derive necessary conditions when (x(a), x(b)) <= E and E is a closed set. 
This completes the proof. 
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